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INTRODUCTION 



Probabilistic Risk Assessment (PRA) is a comprehensive, 
structured, and disciplined approach to identifying and 
analyzing risk in complex systems and/or processes 
that seeks answers to three basic questions: 

What kinds of events or scenarios can occur (i.e., what 
can go wrong)? 

'P' What are the likelihoods and associated uncertainties of 
the events or scenarios? 

What consequences could result from these events or 
scenarios (e.g., Loss of Crew and Loss of Mission)? 
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BACKGROUND 

• The Space Shuttle Program (SSP) initiated the 
development of a Shuttle Probabilistic Risk 
Assessment (SPRA) in March 2001. Prior to that 
there were a number of PRA estimates for the 
Shuttle, but none were sponsored by the SSP. 

— Chart on next page summarizes the Shuttle PRA evolution. 

• The "consequence" or metric of concern selected for 
the SPRA is Loss of Crew and/or Vehicle (LOCV). 

• The risk contributors include hardware failures, 
external events, crew errors, software failures, and 
phenomenological events. 
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SHUTTLE PRA EVOLUTION 



• The advent of established NASA requirements, standards, and tools - as well 
as the development of a strong Shuttle program PRA team have resulted in 
significant recent progress 


• Iteration 3.2 is the most comprehensive and used Shuttle PRA to date 
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BACKGROUND 



The purpose of the SPRA is to provide a useful risk 
management tool for the SSP to identify strengths 
and possible weaknesses in the Shuttle design and 
operation. 

- SPRA was initially developed to support upgrade decisions, 
but has evolved into a tool that supports Flight Readiness 
Reviews (FRR) and near real-time flight decisions. 
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LEVELS OF ASSESSMENT 


• Full Scope SPRA 

— Establishes baseline risk associated with the overall 
mission by mission phase, as well as by vehicle elements 
and subsystems 

- Documented end states, assumptions, approach, and risk 
drivers 

• Focused PRA 

- Answers specific question that doesn't require full model, 
but benefits from it 

• Insights 

— Knowing relative risk contributors provides input for 
decisions without comprehensive PRA 
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KEY INFORMATION FOR MANAGEMENT 

• Clear presentation of analysis 

- if the audience doesn't understand the analysis, the information will 
not be used 

- Difficult because many different ways people process information 

• Applicable assumptions and limitations 

- PRA is only as good as the assumptions that go into the analysis, thus 
important to share for managers to understand the basis of the results 

- Limitations should be understood, so that the results are not misused 

• Estimates of uncertainty 

- state of knowledge about the system being modeled (e.g. the real 
capability of the system to successfully respond to an event) 

- randomness of the probabilistic parameters (e.g. the uncertainty in 
estimating a failure probability of an event) 
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EXAMPLES 
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Shuttle Service Life Extension Program (SLEP) 
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Presenter 


Comparison of Upgrades 


01/20/2004 


Page 



S Assessed the 



Current 
Estimated 
Shuttle Risk (1) 

Current 

Estimated Risk 
Contribution 

Proposed 
Upgrade 
Estimated Risk 
Contribution 

Overall Shuttle 
Risk Estimate 
With Proposed 
Upgrade 

Percent 
Change from 
Current 
Estimate 

AHMS 

1 .28E-02 

1.14E-03 

6.94E-04 

1.24E-02 

-3.5 

AHPS 

1 .28E-02 

1 .22E-03 

4.50E-06 

1.16E-02 

-9.5 

SSME 
CWN (2) 

1 .28E-02 

1 .20E-04 

4.78E-05 

1 .27E-02 

-0.6 

Helium 

APU 

1 .28E-02 

2.34E-04 

9.05E-05 

1.27E-2 

-1.1 


(1 ) Estimate of Loss of Crew / Vehicle risk based on version 1 .5 of shuttle PRA 

(2) Estimates based on values used for Rocketdyne baseline analysis 


risk of each 
proposed 
upgrade and 
compared 
relative 
changes in risk 
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Shuttle Service Life Extension Program (SLEP) 
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Autonomous Shuttle Risk 

Presenter 

Evaluation 

Da ' e 1/26/04 

Page ^ 



Preliminary Shuttle Probabilistic 
Risk Assessment (SPRA) results 
show crew actions during entry are 
a risk driver. 

- Contributions were developed with 
the assistance of the Astronaut Office 
(Dom Gorie). 

- Results / methods are currently 
undergoing an independent review. 

These actions are or could be 
automated, potentially reducing the 
risk of entry. 


S Showed that 
~70% of 
calculated risk 
due to crew 
error occurs 
during entry, 


• Of the approximately 200 crew 

1) 

Crew fails to deploy landing gear 

actions modeled, the top four 

^ 2) 

Crew Brakes at the Wrong Time 

contribute about 11% of the 15% 

W 3) 

Crew Improperly Performs Pre-flare 

human reliability total. 

4) 

Crew Lands too Hard 


descent, and 
landing 


10 


SPACE SHUTTLE PROGRAM 

Space Shuttle Safety and Mission Assurance Office 

NASA Johnson Space Center, Houston, Texas 

Engine Cutoff (ECO) Sensors 



FOUR L0 2 ECO 

POINT 

SENSORS 



CHECKOUT 

COMMANDS 


SHUTDOWN 
COMMAND 
TO SSMEs 


GPC 


ECO 

LOGIC 


POINT 


MDMs 

SENSOR 


ELECTRONICS 


..... 


♦ 0R8ITER 


ORB HER «- 


GROUND 

LOADING 

SYSTEM 


LAUNCH 
PROCESSING 
SYSTEM 


-►GROUND 


FOUR LH 2 ECO 
POINT SENSORS 


S Assessed the risk of 
changing the Launch 
Commit Criteria (LCC) 
for these ECO sensors 
from requiring four of 
four sensors to only 
requiring three of four 
sensors. 

S Pointed out the need to 
better understand the 
other side of the risk 
trade when a launch is 
scrubbed due to ECO 
sensor failures, i.e., 
scrub turnaround risk. 
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Probability of LH2 Low Level Cutoff (STS-122) 




•S Shuttle Program 
Manager requested 
and used 

•S Model used 
historical data in a 
simulation model 

S Shuttle Program 
Manager could see 
it impact of adding 
Ascent Performance 
Margin (APM) on 
risk 
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Solid Rocket Booster Power Bus Isolation Supply Analysis 



Wire Broken at Pin 10 Post 


Destructive Physical Analysis (DPA) 



Critical Pins: 2. 3, 4. 5, 6 


Non Critical Pins: 1, 7, 8, 9, 10. 11 


Median Reliability Derived From 105 No Failure PBIS 
Events 



•S Emphasized the need to 
implement a design change that 
would eliminate the failure in 


PBIS T2 Transformer Leads 


future flights 
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Main Propulsion Flow Control Valve 



INPUTS Thresholds 




5 , 000,000 

replications 


Probability of 

LOCVdueto 

Venting 



S Shuttle Program used these risk estimates as 
supporting flight rationale for STS-119, combined 
with FCV inspection and impact testing 
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Hubble Space Telescope (HST) Manifest Decision 



2.00E-02 


1.80E-02 


1.60E-02 


1.40E-02 


Risk Comparisons 


Expected HST risk is similar to the STS-115 accepted 
mission risk if Crew Rescue is available 

- Reduction in ascent debris risk may partially offset 
the increased MMOD risk for the HST mission 


1:65 


1:77 


1.20E-02 


Ascent Debris 
1:421 

^11% 

1:87 



Ascent Debris 

1.00E-02 





8.00E-03 


MMOD 

1:281 


MMOD 

1:297 

6.00E-03 


Other Ascent 
1:457 


Other Ascent 
1:457 

4.00E-03 


SSME 1:667 


SSME 1:667 



Other Orbit 
1:1230 


otner orbit 
1:1230 

2.00E-03 


Other Entry 
1:402 


Other Entry 
1:402 


32% 


Ascent Debris 
1:392 


MMOD 

1:192 


Other Ascent 
1:457 


SSME 1:667 


Other Orbit 
1:1080 


Other Entry 
1:334 


15% 


V 


STS-115 


2008 Predicted ISS Mission 


HST Without 
Crew Rescue 


Oct 27, 2006 



1:76 

Ascent Debris 
1:821 


MMOD 

1:205 


Other Ascent 
1:457 


SSME 1:667 

Other Orbit 
1:1230 

Other Entry 
1:401 


HSI with 
Crew Rescue 


S Analysis 
compared HST 
risk with and 
without crew 
rescue to other 
Shuttle missions 
in order to help 
NASA 

Administrator 
decide whether 
or not the HST 
mission was an 
acceptable risk 
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Hubble Space Telescope (HST) Manifest Decision (2) 



RISK REDUCTION COMPARISON 



c 

O 

"+■> 

o 

3 

T3 

a) 

a: 

& 

c 

a) 

o 

s- 

d) 
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12 
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8 

6 
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2 

O' 


15% 



Crew 

Rescue 


11% 


SSME Block 
I to Block II 


For an HST mission there are no single 
system changes that would result in a 
mission risk reduction as significant as 
LON/Crew Rescue. 


S Risk reduction 
with crew rescue 
was compared to 
risk reductions 
from 

implemented 
Shuttle upgrades 


i% 



< 1 % 





AHMS 


PGME Added 
to WSB 


«i% 

Wire to Wire Short 
(Inadvertent thruster fire) 


Oct 27, 2006 


HST SM4 Manifesting Review- Pre-Decisional For Internal Use only 
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Probability of Launch on Need 




S Assisted the Shuttle 
Program Manager with 
making an informed 
decision not to release the 
HST rescue vehicle 


PROBABILITY OF NEEDING CREW RESCUE BY DECISION FLIGHT DAY 
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STS-128 Power Controller Assembly Risk Presented at L-2 


STS-128 PCA FAILURE RATE RESULTS 




OV103 

Weibull (3=2.024,1 

=25538) 



S/N | ASSEMBLE | Cycles 

pit) 

5th 

95th 


FPCA-1 V070-763320 

-032/266775 




K1 

AC Inverter 1, Phase A 

127 

4/16/1982 

6100 

1.8E-05 

8.4E-06 

3.3E-05 

K2 

AC Inverter 1, Phase B 

128 

4/16/1982 

6100 

1.8E-05 

8.4E-06 

3.3E-05 

K3 

AC Inverter 1, Phase C 

126 

4/16/1982 

6100 

1.8E-05 

8.4E-06 

3.3E-05 

Kll 

RJDF Bus A 

092 

11/14/1979 

1245 

3.6E-06 

1.6E-06 

6.6E-06 


FPCA-2 V070- 763340 

-013 / J12867 


K1 

AC Inverter 2, Phase A 

096 

1/20/1981 

6300 

1.9E-05 

8.7E-06 

3.5E-05 

K2 

AC Inverter 2, Phase B 

112 

1/20/1981 

6300 

1.9E-05 

8.7E-06 

3.5E-05 

K3 

AC Inverter 2, Phase C 

117 

1/20/1981 

6300 

1.9E-05 

8.7E-06 

3.5E-05 

K13 

RJDF-1 Bus B PWR (RPC#36) 

111 

1/20/1981 

1245 

3.6E-06 

1.6E-06 

6.6E-06 


FPCA-3 V070-763360 

-019 / EJ3166 


K-l 

AC Inverter 3, Phase A 

212 

10/12/1978 

6900 

2.1E-05 

9.5E-06 

3.8E-05 

K-2 

AC Inverter 3, Phase B 

214 

10/12/1978 

6900 

2.1E-05 

9.5E-06 

3.8E-05 

K-3 

AC Inverter 3, Phase C 

215 

10/12/1978 

6900 

2.1E-05 

9.5E-06 

3.8E-05 

K-6 

RJDF-2B Manif F4/F5 Drivers 

216 

12/10/1985 

1245 

3.6E-06 

1.6E-06 

6.6E-06 


MPCA-1 V070-764400 

-039 / ER1634 


K4 

SPARE 

221 

7/11/1989 

700 

2.0E-06 

9.1E-07 

3.6E-06 

K5 

ODS/ECLSS 

228 

7/11/1989 

1180 

3.4E-06 

1.6E-06 

6.2E-06 


MPCA-2 V070-764430 

-033 / F71099 


K4 

SPARE 

103 

3/31/1980 

700 

2.0E-06 

9.1E-07 

3.6E-06 

K5 

ODS/ECLSS 

106 

3/31/1980 

1180 

3.4E-06 

1.6E-06 

6.2E-06 


APCA-1V070- 765310 

-003 / AM6520 


K1 

Reaction Jet Driver Bus A 

138 | 11/10/1982 | 1245 

3.6E-06 | 1.6E-06 | 6.6E-06 


APCA-2 V070- 765320 

-009 / F66222 


K1 

Aft Payload Bay Power B 

137 

3/29/1982 

700 

2.0E-06 

9.1E-07 

3.6E-06 

K2 

RJ DA Manif Drivers Bus B 

180 

2/9/1984 

1245 

3.6E-06 

1.6E-06 

6.6E-06 


APCA-3 V070-765330 

-013 / J43296 


K1 

Aft Payload Bay Power C 

072 

10/10/1979 

700 

2.0E-06 

9.1E-07 

3.6E-06 

K2 

RJ DA Manif Drivers 

079 

10/10/1979 

1245 

3.6E-06 

1.6E-06 

6.6E-06 


Failure rates between 2.0E-06 and 2.1E-05 per cycle 


Probability of a Broken Contactor on STS-128 


Mean - 1:7400 

95 th - 1 :5500 
5 th - 1:10000 


Low Risk due to limited # 
of cycles in flight 


Using a Random failure rate the mean probability of a 
broken contactor on STS-128 is: 1:4100 


Analysis was used 
to help Shuttle 
Managers decide 
that PCA risk was 
acceptable for flight 


Probability of a Broken Contactor on the Ground 

The probability of a SAIL contactor of -15700 cycles > 

old breaking in a 6 week period (Assuming 15 
contactor s and 2 cycles per day) is: ~1 :20 

The probability of a vehicle inverter contactor of -4700 
cycles old breaking in a 6 week period (Assuming 27 
contactors and 4 cycles per week) is: -1 :100 

KEY ASSUMPTIONS 

Assumes 0.5 cycles for AC inverter contactor, 1 .5 cycles for RJD 
contactor and 1.5 cycles for ODS and Payload contactors for STS- 
128 

Analysis assumes failure rate based upon contactor cycles 
5 broken contactor failures are used in the analysis 
Assumes contactor failure will result in inadvertent “off” or failure 
to turn “on” 

Non-latching contactors are not included in the analysis 
Contactor cycles based upon engineering judgment 


Analysis showed 
that it was much 
more likely to have 
a broken contactor 
on the ground 


Important to 
inform 
managers of 
the analysis 
assumptions 


8/23/09 


1 
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STS-131 Helium Isolation Valve Risk 



FAILURE SCENARIO RISK UNCERTAINTIES 



• Given the failed helium isolation valve failed open, the identified risk scenarios have various 
mission impacts as shown in backup chart 6. 

• Loss of Right RCS Function is failure of both regulators and assumes a mission time of 48 
hours (prior to reaching 82% which is expected late FD2, early FD3) and results in NPLS 

• Overpressurization of the Propellant System is failure of both regulators and failure of either 
the burst disc or the relief valve and uses 314 hours (STS-131 mission time) 

• Loss of RCS Control is failure of both regulators and either cross-feed or LRCS failure and uses 
48 hours (prior to reaching 82% which is expected late FD2, early FD3) 

• Each scenario is developed to the point where the mission impact is reached. 

• No change of state in the failed isolation valve is assumed. 

• If both helium isolation valves are assumed to be failed open, the calculated risk for regulator 
fail open will double, which will impact all of the risk estimates. 

2 


S Analysis was 
used to 
support STS- 
131 flight 
rationale at 
the HQ Flight 
Readiness 
Review 
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Right RCS Helium System Reliability 

RIGHT RCS HELIUM SYSTEM RELIABILITY 
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Failure 

I 1st 


2nd 


✓ 


R RCS Fuel 
He Isol 
Fail OP 


Pri Reg 
Fail CL 


OR 


Sec Reg 
Fail CL 


1:1950 Either reg valve failure 314 
hours (STS-131 mission time) 


NEOM 


i 


Pri Reg 
Creep High 

(<Burst Disk) 


>1:100 based 
on flight history 


Pri Reg 
Fail OP 

(>Burst Disk) 


za 

fD_ 

< 

o 

rs 

cn 

rt> 

n 

rD 

era 


u 


Sec Reg 
Creep High 

(<Burst Disk) 




Switch Regs 
A/B 


Loss of Verns 
Impacts on: 
Mated Control 
Mission Content 


Overboard Vent 
thru BD/RV 


1:768 based upon 266 
hours (T-0 to undock) 


1:936 based upon 218 
hours of docked time 


1 :373 using 314 hours 


1 :655 Both reg valves fail, using 314 



Both reg valves failure results in 
Loss of RRCS causing NPLS if 
failure occurs in first 48 hours - 
1:4250 


BD/RV 
Fail CL 


Fuel Tk 
Overpress & 
Structural Fail 


1:93,200 
based upon 314 
hours (STS-131 
mission time) 


Analysis results 
combined with 
graphical display 
to help 

communicate to 
Management at 
HQ Flight 
Readiness Review 
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SUMMARY 


• Showed various ways of communicating and using 
PRA findings in the Shuttle Program 

• Stated that it is important to provide management: 

- Clear presentation of analysis 

- Applicable assumptions and limitations 

- Estimates of uncertainty 

• Maintain consistency and accuracy across the 
program to make it relevant 

• Used various levels of PRA to answer the mail 

• The Shuttle Program has benefited from using PRA 
and others can too 
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